Example-Based Wrapper Generation

Extracting specific information from the vast amount of documents in the World Wide Web is a very tedious task. Manual extraction has high quality output but cannot be automated. Programmed wrappers, on the other hand, suffer from the uncertainty of document structures. The generation of a more generic wrapper for whole classes of textual information, which can accommodate all kinds of document structures, is a crucial problem. Our graphical tool called the Intelligent Tagger allows user to create a grammar composed of rules and patterns which can parse plain text and html documents and retrieve desired information. Users are only required to have knowledge of the information type to be retrieved and its order and structure. With the Intelligent Tagger, grammar creation is performed in three steps: 1) a Graphical Schema Editor helps the user to create XMLSchema definitions with a visual drag and drop interface, 2) an Example Markup Tool allows to markup the desired information in a very simple way, and 3) a Grammar Generator takes the schema and the marked examples and generates a grammar for automatically extracting data from similarly structured documents. This paper focuses on this latter step.